Commentary: Minimizing Evaluation Misuse as Principled Practice

Ethical Challenges, in my experience, is invariably interesting, often instructive and sometimes amusing. Some of the most engaging stimulus scenarios raise thorny evaluation practice issues that ultimately lead to disparate points of view about the nature of the issue and how to handle it (Datta, 2002; Smith, 2002). Despite my poor performance at the tables at the 2003 AEA conference in Reno, I am willing to gamble that the present “ethical challenge” does not fall into the “contentious case” category. I would wager that virtually all of my colleagues would agree: There is really no choice for our fledgling evaluator to make in this situation. She simply cannot proceed as if the problems with the data were minor and could be cloaked to no noticeable consequence. I hasten to add that this is an easy conclusion to reach within the comfort of a hypothetical scenario but an extremely difficult decision for anyone faced with the prospect of returning to a staple of Kraft Macaroni and Cheese Dinners after having lost that long-sought-after job in his or her desired field. And all this, after having slugged it out for so long in graduate school and having invested so heavily in tuition vouchers. Easier said than done, to be sure, yet principled decisions must prevail. The temptation to give in to the allure of personal comfort or questionable employer demands must be resisted. To treat this as a case of “design degradation” that is “a bit worse than most” would be wrong professionally, morally, technically, and probably in several other ways. But concluding what the novice evaluator should not do says precious little about what she should do in this circumstance. Before considering that issue, I want to focus on the client’s, or “intended user’s”1 perspective. Let’s consider a framework articulating evaluation misuse as a possible source of professional practical guidance. Patton has long argued that fostering intended use by intended users is the evaluator’s responsibility. He also proposes that: As use increases so will misuse; policing misuse is beyond the evaluator’s control; and misuse can be intentional or unintentional (Patton, 1988, 1997). Thinking about evaluation misuse, I would offer, has the potential to help the evaluator understand standards of evaluation practice at concrete levels because it can aid in grasping the potential consequences for intended users of her choices and actions. Most discussion of evaluation ethics centers on the actions and behaviors of the evaluator (Christie & Alkin, 1999), yet it is my contention that such considerations are incomplete without serious attention to and analysis of the potential effects of such actions on